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Introduction 


We live in a real world, surrounded by real people & ob- 
jects and interact with them in a real space; however 
most of our digital data and information is restricted 
to a set of metaphors such as keyboard, mouse and 
screen. We have relied majorly on screen based text 
and graphics to interact with computer, specifically 
digital data or information for more than 3 decades. 
Consider it wall mounted screen, hand held device, 
head mounted displays etc. 


Interfaces are not designed to immerse into spaces 
and objects, instead it forces people to interact with 
screen-based metaphors. We as human beings have a 
great ability to perform gestures, speech, remember 
interactions with natural objects; however computers 
do not use these mediums. They ask users to sit on 

a chair, play with keyboard, learn to use mouse, drop 
downs, software to know even a tiny set of informa- 
tion. 


Here comes tangible user interfaces (TUI), where digi- 
tal information is presented through interacting with 
physical metaphor. Physical objects play a central role 
as physical representations and control of digital infor- 
mation (augmenting physical and virtual realities). TUI 
thinks beyond desktop metaphor of mouse, keyboard, 
icon, drop-down, files, folders etc. and embeds informa- 
tion in real world. 
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About 


There is many ways one can define Tangible User 
Interface (TUI), although all of them convey almost the 
same message. Below are the few of them: 


¢ “Giving physical form to digital information that fits 
to our social, cultural and physical context”. 


* “Computation that moves beyond the traditional 
confines of desktop & blends into our daily experiences 
of physical and digital world”. 


- “Augmentation of Physical & Virtual world”. 


- “Integrating information with Everyday Physical 
Objects”. 


- “Mapping physical objects to digital data”. 


Physical environment input becomes an interface to 
interact with digital information as subtle changes in 
color, movement, smell, form, sound, light, audio, video 
etc. 


* Hiroshi Ishii and Brett Ulmer says, 

“Tangible user interface = user interfaces that aug- 
ment the real physical world by coupling digital infor- 
mation to everyday physical objects and environment.” 


Few examples are mentioned below for appropriate 
understanding of above statements. 


1. Marble Answering Machine: 

The physical marbles represent incoming messages. 
The number of marbles that have moved intothe 
pin-ball like chute indicates the number of messages. 
When a marble is dropped into a slot in the machine, 
the recorded message is played. Dropping the same 
marble into another slot on the phone dials the caller 
who left the message. 


2. Urban Planning Workbench (Urp): 

Urban planning workbench uses scaled physical mod- 
els of architectural buildings to configure and con- 
trol an underlying urban simulation of shadow, light 
reflection, wind flow, etc, based on the positions and 
orientations of physical models of buildings, on the 
table surface. 
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3. Music Bottles: 

Music bottles project is a designed table and various 
sets of three corked bottles that “contain” the sounds 
of three instruments or tracks of various musical 
genres. When a bottle is placed onto the stage area of 
the table and the cork is removed, the corresponding 
instrument becomes audible. 


Tangible interaction is an interdisciplinary area. It in- 
cludes a span of several disciplines such as HCl, inter- 
action design, product/industrial design, ubiquitous 
computing, electronics and computer science. 
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Evolution 


The traditional screen based interfaces and disciplines 
like virtual reality, which embedded people into virtual 
world and were seen as estranging people from the 
real world. Additionally, with technology becoming 
cheaper, smaller, smarter and coming quickly to the 
market, it was possible to embed information into 

our surrounding objects, spaces, environment, fabrics 
etc. This motivated the development of the discipline 
called Tangible User Interface. 


The notion of Tangible User Interface constituted an 
alternative vision for computer interfaces that brings 
computing back into the real world, which was origi- 
nally conceived in the mid 90’s. (Wellner, Mackay, Gold 
1993; Ishii, Ullmer 1997).Tangible user interface was en- 
visioned as an alternative to graphical user interface. 
It was proposed to represent digital content through 
tangible objects, which could then be manipulated 
via physical interaction with tangibles. The idea was 
to grasp data with hand and to unify representation 
and control. The thought behind it was to use physical 
devices to interact with digital content. 


In 1996, Fitzmaurice’s PhD thesis had explored the use 
of graspable bricks as a more direct input mechanism 
for the interaction with graphical representations. It 
also suggested the use of multiple graspable objects 
instead of the generic input device such as mouse, 
keyboard etc. An early example of TUI was marble 
answering machine by Durrel Bishop in early 1990s, 
where marbles were used to represent message left on 
answering machine. Placing those marbles plays the 
message recorded in it. 


Similarly, MIT’s Tangible Media Group developed a 
project called Urp (Urban Planning Workbench), where 
they created a map that was manipulated with placing 
iconic representation of central buildings on it and 
moving these apart. The Urp and the more advanced 
Augmented Urban Planning Workbench allowed digital 
simulations of air flow, shadows, reflections, and other 
data based on the positions and orientations of physi- 
cal models of buildings, on the table surface. 


Later, researchers and scientists around the world 
developed interesting projects such as Topbo, Jive, 
Reactable, Microsoft Surface, Siftablesand many more. 
Today almost every appliance contains electronic & 
digital components and becoming smart & intelligent, 
which has formed new challenges and opportunities to 
bring back various design disciplines together. TUI not 
only includes the interface, but aspects such as form 
factor, physical properties of the object, materials, 
technology etc. forced computer scientists, interaction 
designers and industrial designers to focus on design- 
ing complex behaviour that is digitally controlled. 
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Major Considerations 


Major Considerations include: 
4.1 Tangible Input 

4.2 Output 

4.3 Input-Output Technology 


4.1 Tangible Input: 

Tangible input presents the input modalities used to 
interact with information. 

“What kind of medium is used for input? What kind 
of interface we use to interact with information?” 


There are wide range of possibilities to decide the in- 


put modalities depending on the content and context. 


For example, placing a product introduction card on 
the surface must provide information about the prod- 
uct or placing a cube on the table surface. 

E.g. Wall, table, document, 3D objects. 
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4.2 Output: 

Output represents the output possibilities used to 
provide information/ required data to users. 

¢ “What kind of a medium is used for output? How the 
information is been showcased?” 


There are several possibilities to decide the output 
medium to showcase information to users. The output 
may range from a desktop screen to wall-projected 
surface to audio output and mechanical movement of 
3D objects. 
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capture, speech recognition, actuators etc. can be used 
for TUls. 
(Image: arduino, processing, fiducial). 


4.3 Input-Output Technology: 

Input-Output technology represents the importance of 
technology understanding and usage in tangible user 
interface. 


* What is the technology behind the input-output 
interactions? 

¢ “What kind of possible input technologies are availa- 
ble today?” Tibae A} 


Slip 


PLP leet 
Understanding of technology plays a vital role in defin- slieeetanctncdhasu as 
ing interaction modalities. Designer needs to be aware 
of latest possible technologies, their functionality & 
possible usage is designing application. 
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E.g. Use of QR code in advertising. Due to technologi- 
cal advancement, the technological components have 
become smaller, cheaper & smarter to use. Also, de- 
velopment of tools such as arduino, RFID phidgets etc. 
and software such as reactivision, processing, vvvv etc. 
have made hardware prototyping much easier. 


Technologies such as different kinds of sensors, cam- 
era, barcode, image processing, 2D pointing, audio 
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Basic Model 


This section provides understanding of model of user 
interface, model of graphical user interface and model 
of tangible user interface. These models have been 
inspired from models proposed by Professor Hiroshi 
Ishii. These are: 


5.1 Model of User Interface 
5.2 Model of Graphical User Interface 
5.3 Model of Tangible User Interface 


5.1 Model of User Interface: 

Model of user interface proposes two main aspect; 
control and representation. User provides an input 
with actions, controls to manipulate the information; 
based on performed actions digital information is 
been represented and perceived by human senses (E.g. 
Visual, audio, haptic etc.) 


INPUT 


Control + manipulation 


User 
Digital 

B ® Information 
BS | 
85 
& 3 

OUTPUT 

Representation of 
information 


5.2 Model of Graphical User Interface: 

Typically, in graphical user interface, physical controls 
(mostly mouse, keyboard, touch etc.) are used to inter- 
act and manipulate with digital information, while in- 
formation is represented through intangible elements 
such as icons, menu, drop down etc. The information 
representation is perceived by human senses (E.g. 
Visual and Audio). 


 @ 
INPUT ( Control + manipulation ) 
Mouse, keyboard 
(Traditional Gesktop metaphor) / Physical control 
User 
Digital 
Information / Data 


Percieved by 
human senses 
(Visual / Audio) 


OUTPUT 


intangible representation 
( Windows, icons, menu etc ) 


5.3 Model of Tangible User Interface: 

Digital information is represented through intangible 
and/or tangible elements; whereas actions are manip- 
ulated using graspable objects (e.g. cube, theme based 
metaphor such as use of bottles in “music bottle” etc.) 
Immediate tactile feedback is sensed by user when ob- 
ject is grasped and manipulated. Grasped objects can 
have additional computational feedback to enhance 
the responsiveness. Digital information is represented 
through the appropriate use of tangible and/or intan- 
gible elements. 

E.g. Visual, audio, lights, ambience, mechanical move- 
ments etc. 
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INPUT 


Control and manipulation with 
tangible object (3D) 
User 


Percieved by 
human senses 


Intangible representation 
( Windows, icons, menu etc ) 
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Types 


There are mainly two types of TUI: 
6.1 Passive TUIs 
6.2 Active TUls 


6.1 Passive TUls: 

Digital information is represented in tangible and/ 

or intangible form; however participation of the user 
with the product is relatively less. User may not direct- 
ly interact with the product, but indirectly control and 
manipulate the product state. 


Pupspy presents passive tangible user interface. A met- 
aphor of dog is used to represent pupspy that protects 
your computer from virus attacks. Once the virus in 
the computer is detected, pupspy starts barking signi- 
fying the threat. If virus from the system is removed, 
pupspy wages his tail & ears start moving. 


Here, users are not directly interacting with pupspy. 
However, there is an indirect interaction with the prod- 
uct through computers. 


6.2 Active TUls: 

Users are actively involved in interacting with digital 
information. The participation of the user is very high 
& response from the system is immediate. 


Reactable represents active tangible user interface. 
The reactable is a multi-user electro-acoustic music 
instrument with a tabletop tangible user interface, 
which is developed at the MTG in Barcelona. Blocks of 
different shapes are placed on the table top, moving 
them vertically, horizontally or rotating them creates 
music or sound effects. 


In reactable, users are directly interacting with infor- 
mation (in this case music/sound effects) and based 
on users’ actions and manipulation, the contents are 
changing. 
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Key Question 


Mapping of Physical Objects to Digital Computation: 
It is important to understand and choose the appro- 
priate mapping of physical objects with the possible 
feedback. 


Mapping of objects include: 


* Choice of the input object: 
- Why a particular object is chosen as input modality? 
- Is the relationship appropriate between chosen ob- 
ject, its actions & provided output? 

(e.g. using a metaphor of cricket bat & performing 
gestures of hitting a ball should result in a game of 
cricket, not football) 


¢ Placement of the input object: 

- Is the object used for input (control & manipulation) 
placed properly that user understands the next steps 
of interaction? 


There are two ways to choose mapping of the object: 
1. Abstract object: 

The chosen input object may not have any kind of rela- 
tionship with provided output; however, there may be 
a possibility of performed actions associated with pro- 
vided output. Reactable project uses abstract objects 
(cube, triangle, hexagon etc.) to represent music. 


2. Specific object: 

The chosen object have a very clear relationship with 
the provided output, so that user is able to make direct 
relationship between input and output. Here, the 
actions associated may have or not have any kind of 
relationship with provided output. 
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E.g. Urban workbench project uses specific building 
elements & clocks to represent shadow, wind etc. in a 
newly designed architectural space. 


Actions and its Association with Objects: 

actions associated with the objects must provide 
immediate feedback for further steps of interactions. 
Here, actions supported by the objects must be based 
on well-understood actions related to the object. 


E.g. if wine bottle is chosen as input, the opening of a 
bottle cork is a well-understood action 

Appropriate coupling of tangible and intangible rep- 
resentation 


Output of TUI can be represented both in tangible 
and intangible format. It is very essential to under- 
stand the balance and perceptual coupling of tangible 
representation to dynamic intangible representation. 
Tangible elements should be over shadowed or even 
should not be invisible. 
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Contact Details 


This documentation for the course was done by 
Professor Keyur Sorathia, faculty at DoD, IIT Guwahati. 


More details about his research can be found on 
www.embeddedinteractions.com 


You can get in touch with him at keyur[at]iitg.ernet.in 
or keyurbsorathial[at]gmail.com 


You could write to the following address regarding 
suggestions and clarifications: 


Helpdesk Details: 

Co-ordinator 

Project e-kalpa 

Department of Design 

Indian Institute of Technology Guwahati 
North Guwahati 

Guwahati 781039 

Assam, 

India 


Phone: +91-361-2582500, +91-361-2582451 
Fax: +91-361-2690762 
Email: dsource.in[at]gmail.com 


